home
***
CD-ROM
|
disk
|
FTP
|
other
***
search
/
Best of www.BestZips.com (Collector's Edition)
/
Best of WWW.BESTZIPS.COM Collector's Edition (JCSM Shareware) (JCS Marketing).ISO
/
teachtol
/
truemc40.zip
/
MANUAL.TX_
/
MANUAL.TX
Wrap
Text File
|
1997-01-29
|
33KB
|
783 lines
------------------------------------------------------------
TRUE MULTIPLE-CHOICE INSTRUCTION MANUAL 0/14
INDEX
------------------------------------------------------------
WRITE, EDIT and PRINT
Create and Enter Questions . . . . . . . 1
Select and Edit Questions . . . . . . . . 2
Print Cram Notes and Files . . . . . . . 2
Format and Print Tests . . . . . . . . . 3
Problem Questions . . . . . . . . . . . . 4
------------------------------------------------------------
REVIEW, TUTOR and TEST
Cram, Practice and Test . . . . . . . . . 5
Levels of Questioning . . . . . . . . . . 6
Test Scores . . . . . . . . . . . . . . . 7
Test Record Sample . . . . . . . . . . . 8
------------------------------------------------------------
BIBLIOGRAPHY
Asking the Questions (1980-1990) . . . . 9
The Answers (1991-1994) . . . . . . . . . 9
Application in the Classroom (1995-) . . 10
------------------------------------------------------------
WORKSHOP
Higher and Lower Levels of Thinking (LOT) 11
Question Performance . . . . . . . . . . 11
Good Questions . . . . . . . . . . . . . 12
Question Development . . . . . . . . . . 12
True Multiple-Choice Edit Window . . . . 12
True Multiple-Choice Tasks . . . . . . . 13
Practice Doing Something Useful for You . 13
Educational Research Laws . . . . . . . . 13
Workshop Sample Applications and
Support Tasks . . . . . . . . . . . . . . 14
------------------------------------------------------------
Task #1: Write and Enter Questions 1/14
The easiest and the most difficult of the five tasks is
to create a question file. The easiest way is to copy from
old tests or old test banks. The most difficult way is to
create new questions. The original program was created to
help rapidily produce high quality questions from current
instructional materials. Eventually the method was recorded
as a guide for students. It then became the Edit screen in
this program
A QUESTION IS ENTERED IN PARTS. The minimum is the
question stem and one answer (short answer) or two answers
(multiple choice). There is space for two right and four
wrong answers. Each file is limited to 200 questions.
QUESTION FILE NAMES are limited to eight characters
plus a three character extension (BIOLOGY1.TXT). Some
thought should be given to how files are to be named. One
method is to use a separate disk for each course, and name
the files for text chapters or for unit tests.
This TASK CAN BE INTERRUPTED at any point. On
returning, the old file can be picked up again or the new
questions can be appended to the old file. If the old file
is picked up, the first screen shows the last question in
the file. Questions in the file can be located by a FIND
FUNCTION.
The ERASE FUNCTION only marks a question for erasing.
It can be retrieved by the Find function which will
display all items marked for erasing.
This task was specifially designed for COPYING OLD
QUESTIONS. It can be assigned to anyone with a minimum of
instruction (see Problem Questions, page 4). The screen
changes color when the starting space is exceeded. Part of
the item may scroll out of view.
When CREATING QUESTIONS, the task allows one to enter
questions as they occur, in incomplete fragments, which can
be completed later. The next task, Select and Edit, has
even greater flexibility for creating and proofing
questions after the initial file is created with this task.
Review and Test can also be used to view questions.
However, it does not permit editing questions. Instead,
make notes of changes, and make them with task #2,
Select and Edit.
In general, the function to be repeated in a task is
assigned to the <RETURN> key. This increases productivity
once you have become familiar with the task.
Task #2: Select and Edit Questions 2/14
This task was originally used to select questions from
several question (data) files into a pool. Then several
different versions were formated from the pool.
Change the order of the questions with MOVE. New
questions are added at the end of the file. Place them
where you want them with MOVE.
Again the <RETURN> key is assigned to a repetitive
function: PASSING. Each tap of the key shows the next
question in the file. The SELECT FUNCTION places a question
into the new file of pooled questions. The REJECT FUNCTION
drops it from being viewed again.
When working with high quality question files, the
automatic feature saves much time. For example, a 50
question test pool can be randomly drawn 10 questions each
from 5 files in a few minutes. This test then becomes a
sub-sample of all of what could have been asked. It is a
sub-sample of what students were expected to know or be able
to do. The score is an indicator of the scores of other
possible tests created from the same question files.
Question quality can be determined statistically and
by inspection. Are there two good acceptable answers? Do
the wrong answers include options commonly associated with
the stem or that are closely related? Do the answers
include common misconceptions? This becomes the rubric for
scoring student authored review question files. Editing and
scoring follow the same rules.
STUDENT: Create by writing question stems that
describe what you know or can do. Your
understanding. Add a key term or phrase
for a right answer. Add closely related
unacceptable terms and misconceptions
for wrong answers.
TEACHER: As an assessment, score with the above
rubric. To make class paper tests,
select and edit from student created
review question files. Start assignments
with a file of descriptions that students
then complete and put in order (edit).
Task #3: Print Cram Notes and Files
Printed data files can be edited anywhere and at any
time by faculty and students. Each question part is labeled
to match the editing prompts. When just printed to the
screen, this makes a quick review.
Task #4: Format and Print Tests 3/14
This task transforms a data file into a text file for
printing out on paper. If the questions have been pooled,
all that remains to do is to select the data file, select
automatic, and the text file name (TEST1.FIL). In a few
seconds, a new test is created ready for printing. A second
formatting from this file will produce a test (TEST2.FIL)
with the same question sequence but with the answers in a
different order from the first test.
However, many faculty members like to pick and edit
each question individually (concretely). This task can be
performed by drawing questions from one or more data files.
As each question is presented, it can be selected, rejected,
edited, or passed. If selected, the formatted question can
also be selected, rejected, edited or passed. Editing the
DATA base makes a permanent change, editing the FORMATTED
item changes the question for the current test only.
A record is kept of the number of questions selected
from each data file. As the program ends, you will be asked
to confirm if changes to the data base are to be made
permanent.
When finished selecting, the formatted questions are
printed to a file followed by the answer key. This file can
be printed out as the test or picked up with a word
processor to add other instructions and illustrations.
Each time a question file is formatted into test
questions, the answers are randomized. Problem questions
generate new answer options. Each formatting produces a
different practice or real paper test.
Print the formatted file with your favorite wordprocessor,
Notepad, or Wordpad. These permit inserting page breaks and
space for illustrations. A simple way to handle illustrations
is to prepare them in a separate handout that is inserted into
the test. Questions then cite the illustration on the insert
or any other source or observation available in the room.
Generalized illustration inserts can be reused on many tests.
The pooled data file can also be picked up with Review,
Tutor and Test and used as a practice or makeup test.
Problem Questions 4/14
Questions that require a numerical calculation can have
many different answers if question and answer codes are used
in the question. All values except one are held constant.
The variable value is given as a range. Each time the
question appears, the variable value is selected at random
and new answer options are presented. Example:
Q1 A wave with a frequency of 10 hertz is moving how fast
if the wavelength is **2-100 m?
R1 **10 m/s
R2
W1 */10 m/s
W2 /*10 m/s
W3 *+10 m/s
The question variable range code (**##-##) is **2-100.
Values from 2 through 100 will be selected at random and
then modified by each answer code. In the example, the
constant is 10 from 10 hertz. The constant can be derived
from more than one value in the question (as from a
formula).
Answer codes: ** variable * constant multiply
*/ variable / constant divide
/* constant / variable divide
*+ variable + constant add
*- variable - constant subtract
-* constant - variable subtract
(The problem and resulting question can be viewed together
with Select and Edit, Task #2.)
Non-randomizing Answers
To make True Multiple-Choice compatible with
traditional multiple-choice questions furnished in test
banks, the following limitations have been made:
True and False True always prints before false.
All of the above One or both options print last and
None of the above in this order.
**L as first answer L is the location (A - E) of the
(R1) right answer in a fixed set of
non-randomizing answers listed in
consecutive locations R2 to W4.
Cram, Practice and Test 5/14
These program features were designed in response to
needs expressed by students who were doing poorly in general
education classes. All students can CRAM. Next they can
PRACTICE. Finally they are ready to take a real practice
TEST. This also reduces test anxiety.
1. CRAM is a SEQUENTIAL presentation of questions in a
rational order. Items answered incorrectly are repeated.
They must be answered twice correctly to end the
repetition. The goal is to familiarize the student, to
complete a review of the assignment in a reasonable
amount of time.
2. PRACTICE is a RANDOM presentation of questions. It is a
practice in recognizing questions as a sample of the
body of related questions. Items answered incorrectly
are repeated.
3. TEST is like PRACTICE except items answered incorrectly
are not repeated. Now is the time for the student to
report what is known or can be reasoned on the first
passes through the test. Correct answers are shown if a
question is answered incorrectly.
Two scores are printed. One is the traditional score
based on knowledge (Kn). The other is based on
knowledge and judgment (KJ) where judgment is given the
same weight as knowledge. The knowledge and judgment
(KJ) score and the knowledge (Kn) score are the same
if the test is completed in one pass. If completed in
more than one pass, the knowledge and judgment (KJ)
score is its value at the end of the next to the last
pass, before the final pass using random guessing.
Review, Tutor and Test has been Used Three Ways
1. To help students survive until they can question and
learn without its help.
2. To help students abandon the passive concrete level of
thinking and discover the active self-correcting level
of thinking needed to understand and do science (and
other things) in school and college.
3. To help build confidence to the point students can write
a record of their questions as they makes sense of an
assignment: a review question file.
The final step is for students to write a set of class
notes that make sense. The Write, Edit and Print portion of
this program, also used by teachers to write tests, provides
one means for doing this. The review question file becomes
of greater value with each use when students edit (read
critically) as they review.
Levels of Questioning 6/14
Each question is presented at the highest level of
thinking. If you cannot answer, press the "ENTER" key for
the next lower level of thinking. At the bottom, one more
press will display a right answer during CRAM and PRACTICE.
Student: (4)
Top students prefer to enter their own answers rather
than select from multiple choice options. The
answer needs to be short and unique. Something that is
a part of a right answer but not of a wrong answer.
The entry must be a minimum of five characters long, or
the complete right answer if shorter.
The query, biolo, will find biology and biological. The
query, water AND hot, will find hot water but not water
alone. The query, 1 OR one OR first, will find an
answer in three different ways.
You do not need to memorize the answers. You do need
to classify and think of alternative answers.
Scholar: (3)
This produces startling effects on first time users.
The question vanishes as the answers appear. You must
have read the question with sufficient understanding to
answer without referring back to the question. This is
very much like an oral examination. Practice
visualizing the question.
Scientist: (2)
Enter a false answer. It will be erased. Continue
until only one answer remains, the true answer.
Falsifying answers is a powerful way to study. Rather
than having one key term related to one question stem,
you will know several key terms that can be the correct
answers to several questions.
Pupil: (1)
Marking the correct answer option is the traditional
way of taking multiple-choice tests.
Tourist: (0)
If you cannot recall or reason an answer, good judgment
is to pass. An acceptable answer will appear during
CRAM and PRACTICE. Consider why it is a correct answer
before continuing. You may need to review class notes,
do more reading, make more observations, or try a
different line of reasoning.
Test Scores 7/14
The BEST TEST SCORE is obtained by working through the
question file more than once. Each time select more
difficult items:
1. Answer with confidence.
2. Use easy educated guessing.
3. Use hard educated guessing (guess from two options).
4. Use random guessing on the last pass.
Press PROG for a PROGRESS REPORT at the end of each
pass through the file or after every 10 or so questions
answered. The information is presented in a graph and a
table. If good progress is being made, keep on selecting
more questions to answer. If not, it is time to pass when
in doubt.
The NEW PROGRESS REPORT at the end of the test will
print to the screen, to a printer, or to a file. If you
need the record, you must print it to a printer or to a
file (NEWPROG.TXT). A Test Report sample is on the next
page. It contains an progress graph and a diagnostic
sequence list of the questions answered.
This student did very well on the first 36 questions.
Pass number seven, the last pass, is the random guessing
pass. If the student had declared random guessing on the
second pass instead of the seventh, the knowledge and
judgment (KJ) score would have been 67% instead of 62%.
This program selects the KJ score as the value prior to
the last pass (the random guessing pass) through the test.
It can also be located by inspection of the progress graph.
The highest point on the knowledge and judgment plot can be
assigned the KJ score. This would be a very liberal
valuation of judgment.
Scoring for judgment rewards students for reporting
what they know or can reason. This shifts the
responsibility from the teacher to the student both for
knowing and for having a sense of judgment. The workplace
also rewards both knowing and judgment. Knowledge and
judgment scoring is an "alternative assessment" that is
fast and inexpensive. You know what students know because
they are free to tell you.
The KJ score can be obtained from paper tests also.
This requires having students select only the questions they
can answer with confidence or educated guessing. If both
KJ and the traditional knowledge (Kn) scores are desired,
two answer sheets (or a combination sheet) are used.
The OLD REPORT RECORDS file keeps general information.
Click on FACTS for OLDREPRT.TXT data descriptions.
Test Report Sample 8/14
Report for: Sample Report
Question File: SFAA4.IQF at 19:07:50 on 09-21-1993
SCORING SEQUENCE WITH 31 RIGHT AND 19 WRONG
----------------------------------------------------------
. . . . .
. . 2 . . .
. . 2 2 2 . .
. 70 . R E C I T I2N G2 2 . 70%
. . . S E N S2E . 2 .
. . . 2 . 2 .
. . . 1 . 2 .
. . . 1 . 2 . Knowledge and
. 65 . . 1 . 2 6 . <--Judgment
. . . 1 . 2 7 . (KJ) Score
. . 1 . . 7.
. . 1 . . ***7 <--*-Knowledge
. . 1 . . .. (Kn) Score
. 60 . 1 1 . ******** . 60%
. . . 1 1 . . ..
. . . 1 . . *** . .
. . . 1 . . . .
. . . 1 . .* . .
. 55 . . 1 . . . . 55
. . 1 1 . * . . .
. . 1 1 1 . . . . .
. . 1 1 1 1 . . * . . .
.. 1 1 1 . . . . .
50* 1S1T A R T I N G . *G U E S.S I N G . 50%
1 1 . . . . .
. . . . . . . . . . . . . . . .*. . . . . . . . . .
0 10 20 30 40 50
THE TOTAL NUMBER OF ANSWERS ENTERED SCORE
----------------------------------------------------------
Question File Numbers for the Above 50 Answers and
the Number Answered in Each Pass Through the File
----------------------------------------------------------
1 2 3 4 5 6 7 8 9 10
----------------------------------------------------------
0 15 46 24 35 22 4 26 25 32 2
P#1-----------<know and can do>-----------------
10 47 10 28 38 16 33 8 41 48 6
--------------------------------------------------
20 14 1 45 44 9 5 17 37 12 49
--------------------------------------------------
30 11 27 42 7 13 43 50 29 36 20
>(31) P#2------<educated guessing>---------------
40 23 34 30 3 31 40 39 21 19 18
------------------------->(15) P#6 P#7----->( 3)
<random guessing>
----------------------------------------------------------
Asking the Questions (1980-1990) 9/14
Hart, Richard A. 1981. Evaluating and rewarding student
initiative and judgement or an alternative to "sitting
through" a course if you did not test out. Pages 75-76
in Directory of Teaching Innovations in Biology.
Meeth, L. R. and Dean S. Gregory, Ed. Studies in
Higher Education:Arlington, Virginia. 252 pages.
Hart, Richard and Kenneth Minter. 1985. Using a
computer to manage typical classroom problems.
National Science Teachers Association Annual Meeting,
Cincinnati, Ohio 18-21 April.
Minter, Kenneth and Richard Hart. 1986. Essay testing
using multiple choice questions. Missouri Academy of
Science Annual Meeting, Warrensburg, MO 25-26 April.
Hart, Richard and Kenneth Minter. 1988. Diagnostic
Testing Using Multi-Choice and Matching Questions.
National Science Teachers Association Annual Meeting,
St. Louis, MO 7-10 April.
Minter, Kenneth and Richard Hart. 1989. Student Choice
in Computer Graded Tests. National Science Teachers
Association Annual Meeting, Seattle, Washington
6-9 April.
Hart, Richard. 1989. Practical Programs for Oversized
Classes Resulting from Investigations into the Use of a
Campus Computer System as a Part of an Instructional
System (1981-1989). 217 pages. TX 2 602 807
Hart, Richard. 1989. Professor of the Year. Northwest
Missouri State University.
Hart, Richard. 1990. Professor of Biology, Emeritus,
Northwest Missouri State University.
The Answers (1991-1994)
Hart, Richard. 1991. Founded Nine-Patch Software to
underwrite the distribution of research results.
(Nine-patch is the quilt design created from the three
levels of thinking used by the three parties involved
in education: students, teachers, and administrators.
The nine squares represent the nine ways anything
said or done in a classroom can be interpreted.)
Hart, Richard and Kenneth Minter. 1991. Student Choice
in Multiple-Choice Testing. National Science Teachers
Association Annual Meeting, Houston, Texas
27-30 March.
(continued) 10/14
Hart, Richard. 1992. Publication of quality scoring
research results in three practical computer DOS
programs. TRAINER for class tests, EQUALIZER for
individual interactive use, and OLYMPIAD, a specific
contest scoring application of TRAINER.
Hart, Richard. 1993. Founding board member and volunteer
treasurer of the Educational Software Cooperative, Inc.
The ESC publishes educational software on CD-ROM and
assists authors and teachers on CompuServe, Edforum,
Section 2.
Hart, Richard. 1994. EQUALIZER (DOS) edited into
EXPEDITOR (VB for DOS and Windows).
Application in the Classroom (1995-)
Motes, Andrew. 1995. ESC founder, features both
traditional scoring for ranking and scoring for
knowledge and judgment in the most recent verions of
the award winning children's educational software
"Best of 1994", School-Mom Plus.
West, Rosemary. 1995. ESC home page: http://members.aol.
com/edsoftcoop/
Hart, Richard. 1995. TRAINER (scores knowledge and judgment)
and True Multiple-Choice (write, review, tutor and test)
released for middle-school, high school and college.
Hart, Richard. 1996. Scoring paper tests traditionally and
by knowledge and judgment; student test on a disk; and
writing questions and printing tests. Workshop,
including department and campus network site licenses,
20 Aug. Department of Nursing, Southeast Missouri State
University.
Hart, Richard. 1996. Converting class notes into review
question files and converting review question files into
class tests. Workshop, including individual licenses
for student participants, 27 Sept. Department of
Nursing, Southeast Missouri State University.
------------------------------------------------------------
Nine-Patch Software 11/14
(Scoring by Knowledge and Judgment)
Testing Workshops
by
Dr. Richard A. Hart, Ph.D.
Professor of Biology, Emeritus
Northwest Missouri State University
Index
-------------------------------------------------
A. Higher and Lower Levels of Thinking (LOT)
B. Question Performance
C. Good Questions
D. Question Development
E. True Multiple-Choice Edit Window
F. True Multiple-Choice Tasks
G. Practice Doing Something Useful for You
H. Educational Research Laws
I. Workshop Sample Applications and Support Tasks
-------------------------------------------------
A. Higher and Lower Levels of Thinking (LOT)
------------------------------------------------------
Boundary Area Synonym Relationship
------------------------------------------------------
2. Problem Solving (cyclic, self-correcting)
Higher Understand Work With
1. Concrete (linear, attentive)
Lower Memorize Work On
0. Random Guessing (unresponsible)
------------------------------------------------------
(Liberal Arts Student, Pupil and Tourist or health
care personnel with different types of training.)
B. Question Performance
-------------------------------------------
How a question performs depends upon:
-------------------------------------------
1. The instructional system LOT.
a. Copy notes to be memorized.
b. Assignments requiring understanding.
2. The student performance LOT.
a. When studying.
b. When testing and reporting.
-------------------------------------------
C. Good Questions 12/14
-------------------------------------------------------
1. Are simple and direct (no need to be clever).
a. Stem is a description or problem.
b. Options are labels or answers that are
within and outside of acceptable limits.
2. Can be used for review, instruction and evaluation.
-------------------------------------------------------
D. Question Development
---------------------------------------------------
Author: Student Teacher
---------------------------------------------------
Question: Stem Stem
Right Answer: 1* 1 or 2
Wrong Answer: 0* 0 to 4
Returns as: Short Answer Short Answer
(flash card) Visualized
Scientific
Multiple-Choice
Random Guess
---------------------------------------------------
* Can be supplemented by student or teacher.
E. True Multiple-Choice Edit Window
---------------------------------------
Q1> (stem is required)
R1> (1 right answer is required)
R2>
W1> (0 to 4 wrong answers)
W2>
W3>
W4>
---------------------------------------
Permanent note pad at bottom of screen.
---------------------------------------
F. True Multiple-Choice Tasks* 13/14
----------------------------------------------------
1. Author Tasks: Write, Edit, and Print
2. Instructional Tasks: Review, Tutor and Test
3. Program Mode: Read and Write or Read Only
4. Other Useful Stuff: Demonstration and Manual
----------------------------------------------------
* The companion program, TRUE SCORE TEST GRADER,
scores class tests both traditionally (answer all)
and by knowledge and judgment. It can also assist
in assigning grades based on the valid questions
in the test.
G. Practice Doing Something Useful for You*
----------------------------------------------------
1. Creating a test from existing test banks.
2. Writing test questions in small groups
(teachers, teachers & students, and students).
a. Individual questions.
b. Situation or linked questions (citing).
c. All "right answer", instructional questions.
3. Writing review questions in small groups.
4. Editing review questions into test questions.
5. Scoring by knowledge and judgment.
a. Program demonstration.
b. By hand.
----------------------------------------------------
* Each activity can take from 15 minutes to an hour
depending upon group interest.
H. Educational Research Laws
------------------------------------------------
1. Don't do it. Find someone else. They will
get the blame for the premature or faulty
implementation of the research results.
2. Don't tell people they need to reform.
Help them do what they are already doing in
the direction you have in mind. You help
them and they help you (quality management).
------------------------------------------------
I. Workshop Sample Applications 14/14
These single task applications were written for workshops.
After carrying out the task, the participant should be able
to comfortably continue doing the task.
------------------------------------------------------------
Workshop Sample Applications
------------------------------------------------------------
1. Write questions and save in a file.
2. Select questions for a test.
3. Format and print a test.
4. Tutoring with an instructional question file.
5. Student test on a disk.
6. Multiple forms of one test.
7. Converting student lecture notes to review questions.
8. Class and group writing.
9. Converting review question files into instructional
question files (and tests).
10. Tutoring and tutoring services.
------------------------------------------------------------
These support tasks answer questions commonly asked during
workshops. There is no need to know Windows or MicroSoft
Word before using the program.
------------------------------------------------------------
Support Tasks
------------------------------------------------------------
1. Converting old test files to True Multiple-Choice files.
2. Using Windows 3.1 to read, print, and delete files.
3. Using the license key.
4. Spell checking and editing question files.
5. Margins, page breaks and printing tests.
6. Location of question and text files.
7. Non-randomizing answers.
------------------------------------------------------------
11/96
